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Sexual References 
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Pseudo-science and/or science denialism 


Australian politics 

































































































11:30 AM-13 Oct 2017 


@HamillHimself0 

©Harnill Himself 

Early Focus-Group scores indicate that the 
chances of Episode IX being called: "FROM 
HIS NAP" are extremely low for some reason. 

#TepidTitle 














Emma Watson O 

@EmmaWatson 


@bestfriends @BuzzFeed @beourguest 
#BeautyAndTheBeast 


• % 


L" « 


Emma Watson Plays With Kittens (While Answering Fan Qu. 

a l want them all!" GET MORE BUZZFEED: 
https://www.buzzfeed.com https://www.buzzfeed.com/videos 
https://www.youtube.com/buzzfeedvideo https://www.youtub... 

youtube.com 


11:19 AM - 16 Mar 2017 



Kristen Bell 0 

®IMKristenBell 


Overheard my daughter tell another kid:"If ur 
parents ever get blown away in the wind, just 
pick up the phone & call 891." 

#SafetyFirst 


10:28 AM-6 Nov 2017 


■ < »>■■■«■■ mi 
































































































































SwiftOnSecu rity 

@SwiftOnSecurity 


Replying to @MalwareTechBlog 


I was Taylor Swift and now I’m Taylor Swift 
with a job in IT 

4:41 PM-8 Oct 2017 




































Anthony B, f 

@swearyanthony 


Cassowaries - come on you're not even a 
bird, you're a collection of grievances and 
hate because you are a failed dinosaur. 

1:50 AM -20 Nov 2017 
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Sarah Gailey 

@gaileyfrey 


Who wants to hear the story of the date 
witnessed between Wolf Girl and James 
Spader's Lonely Doppleganger? 


I HOPE IT'S YOU 


5:44 PM-25 Oct 2017 



■■1 















































































The Onion O 

@TheOnion 


Discovery Of Neolithic Gift Shop Suggests 
Stonehenge Always Meant As Tourist 
Attraction trib.al/swFqUbC 



6:19 AM -30 Nov 2017 











































Tim Cook Q 

@tim cook 


Congratulations Australia! Another important 
step toward equality for all. 

#MarriageEquality 



4:32 PM -14 Nov 2017 



Elon Musk O 

@elonmusk 


Payload will be my midnight cherry Tesla 
Roadster playing Space Oddity. Destination 
is Mars orbit. Will be in deep space for a 
billion years or so if it doesn’t blow up on 
ascent. 

6:22 PM -1 Dec 2017 
























































































Barack Obama O 

©BarackObarna 


"No one is born hating another person 
because of the color of his skin or his 
background or his religion..." 





Km 


IM 


— 


1 


5:06 PM - 12 Aug 2017 























































































Donald J. Trump 

@realDonaldTrump 


My Twitter account was taken down for 11 
minutes by a rogue employee. I guess the 
word must finally be getting out-and having 
an impact. 

3:51 AM -3 Nov 2017 




































|Christopher Neugebauer 

^ ©chrisjrn 


I hate t-shirt shirt manufacturers. 



7:07 PM-14 Nov 2017 







































































































Sam zlbfi-Kimbrel 

@skimbrel 


Replying to 


“Writing unit tests in C is probably an order 
of magnitude more painful than writing the 
implementation code” 


Hang on, where’s 


11:27 AM - 3 Dec 2017 from 


? 



































r- 1 



r XJv Git Commands 



— | 




f-J 


t *5* 


Midsomer Murders Bot 

@midsomerplots 


A high flying airline pilot is 


Accidental Haiku 

@accidental575 


Whv mus t I he cursed/ 






Magic Rea 

@MagicRealisn 


™i 



A 

P 

P 


2:C 


Appropriate 

.ii7)aod.t£ih.utos_ 



Appropriate Tributes 

@godtributes 


Replying to @curlingup 


BOTS FOR THE BOT GODDESS 


th© aruCH 4:54 PM - 1 Nov 2017 

® ® I____ 

@thegrugq_ebooks 


—i— 


Possibly the best off the shelf solutions to 
monitor and cyber secure that basket!!! 

10:02 AM -27 Nov 2017 


1-MtKGEJT-Q TIP] 
s and save them to 


red 



3:00 AM -15 Nov 2017 


mm* 

























































































































































































@pentametron 


i 


mika * 

* @phantasticdrwho 

i want a giant rooster for a pet 

2:59 PM-13 Aug 2017 


w 



Jordan Stump 

@jayyrene02 


Why doesn't Taco Bell deliver yet£ 

7:28 PM-13 Aug 2017 


t 













































@pentametron 



Jimmies = Rustled 

@KeyboardCowboyy 


Eggs Benedict Appreciation Tweet 

6:14 AM -29 Nov 2017 



w 



bree van de kamp 

@justsay_shay 


Delete delete delete delete delete! 

1:55 PM -29 Nov 2017 













































@accidental5 7 5 


a ..... | Accidental Haiku 

1 @ accjde ntal575 


1 ..... > Accidental Haiku 

@ acc j denta | 5 7 5 

i only accept / 


So Javert, you see / 

apologies in the form / 


it's true. This man bears no more / 

of karaoke / 


guilt than you! #Valjean / 

#accidentalhaiku by ©kennfitzz 


#accidentalhaiku by @LesMis_bot 

kenn £ @kennfitzz 


LesMiserables Lyrics @LesMis_bot 

i only accept apologies in the form of karaoke 


So Javert, you see it's true. This man bears no more guilt than you! #Valjean 

8:00 AM-4 Nov 2017 


8:01 AM -18 Nov 2017 

























@StealthMountain 


n» 


Jewel Staited ©JewelStaite * 29 Apr 2016 

Want a sneak peak of my episode of #Castle airing 

Castle 


© 

WM 


\ M- - 

A 17 V 

1:18 

£>c 

fAi 


Q 85 


tn 248 Q? 1.2K S 


« Stealth Mountain 

©StealthMountain 
Replying to ©JewelStaite 


Follow ; v* 


©JewelStaite I think you mean "sneak peek" 


12:01 PM -29 Apr 2016 



4 ' 



Bill Gates O ©BillGates • 14 Dec 2014 
I loved The Rosie Project. I got a sneak peak at the 

gat.es/12Zj3AN 



* 


Q 122 O 456 Q 902 

Stealth Mountain 

©StealthMountain 


0 


( Follow 


s/ 


Replying to ©BillGates 


©BillGates I think you mean "sneak peek" 


2:29 PM-14 Dec 2014 











































@fewerror 


Account suspended 

This account has been suspended. Learn more about why Twitter suspends accounts, or return to your timeline. 






































@godtributes 



% Tmore stuffing please i % @oldtobegin • 16m 

the best part of shows about hoarding is the 3 PM spaghetti dinner at the end of 
the show 

Q 1U V 0 

Appropriate Tributes 

@godtributes 


Replying to @oldtobegin 

SPAGHETTI FOR THE SPAGHETTI 
PRESIDENT! HOARDING FOR THE 
HOARDING GODDESS 


10:00 AM -29 Nov 2017 

—1^—IM1IM^^1— mm - 1 — —■ m ——-n ■ 



































































































@godtributes 



Cezar Augusta ©cezaraugusto • 1h 

macOS High Sierra security update for root login problem was addressed 




About the security content of Security Update 201... 

This document describes the security content of 
Security Update 2017-001. 

support.apple.com 


Q i IT C? i B 



Appropriate Tributes 

©godtributes 


Replying to @cezaraugusto 

| | A | | FOR THE ft GOD! 
SIERRAS FOR THE SIERRA HOME PLANE! 

8:51 AM -29 Nov 2017 























































































Charles Bergquist© ©cbquist • Nov 17 

Our special guest for this episode of @scifri is Terrifying Cabbage Dog 




Q 7 tl 7 o 63 B 


Appropriate Tributes 

©godtributes 
Replying to ©cbquist 

TOYS FOR THE TOY GOD! SNOUTS FOR 
THE SNOUT GODDESS 

10:55 AM - 17 Nov 2017 











































































































@godtributes 


♦ 



W 



Appropriate Tributes 

@godtributes 


BLACK MARIA FOR THE BLACK MARIUM 
GOD 

5:00 AM-26 Nov 2017 









































@MagicRealismBot 


Magic Realism Bot 

@MagicRealismBot 



A schoolmistress watches a movie about an 
SQL injection that can destroy the Sun, and 
sets out to find it. 


11:18 PM -23 Nov2017 


* Magic Realism Bot 

@MagicRealismBot 



A factory in Silicon Valley produces vast 
quantities of nihilism. 


7:18 AM -5 Nov 2017 










































Toxicology says she was high on somethin' 
called Bathtub Boosters. It's a tranquilizer for 
polar bears. 


Toxicology says she was high on somethin' called 
Bathtub Boosters. It's a tranquilizer for polar bears. 


3:00 PM - 13 Nov 2017 



Fin 

®icetsvu 


Kids are callin' it Sicilian Grout. He set a 


dumpster on fire behind a falafel shop and 
huffed the fumes until he thought he was 
gettin' attacked by an army of giant adult 
coloring books. 



on fire behind a falafel shop 
until he thought he was 
giant adult coloring books. 


3:00 AM -20 Nov 2017 




















































midsomerplots 


•O Midsomer Murders Bot 

©midsomerplots 

A local mushroom hunter is found drowned 
in cider. Suspicion falls on the village 
prodigal son, angry that the metric system 
might threaten the annual cheese festival. 

10:53 PM - 18 Nov 2017 


a Midsomer Murders Bot 

fl ^ • 

@ rn 'dsomerplots 


A visiting Toby Jug provenance evaluator is 
found stuffed full of battenberg. Suspicion 
falls on the village chapter of the Ayn Rand 
society, angry that a museum dedicated to 
an obscure local poet might threaten near- 
lethal levels of tweeness. 


2:53 AM - 18 Nov 2017 




























https://github.com/matthewberryman/midsomerplots 


var murdered p erson = [ 

"A local bee keeper" , 

"A socially-inept Twitter bot developer" , 

"A visiting Toby Jug provenance evaluator" , 

"The town's cantankerous children's party planner" , 
"A local archaeologist' , 

"A chilled out yoga instructor 8 , 

"A seditious Marxist", 

"A digital content person", 

"A retired American woman" , 

"A lonely librarian", 

"An underappreciated archivist' , 



































https://github.com/matthewberryman/midsomerplots 


var cause_of_death = [ 

"entombed in a statue", 

"bowl'd to death with turnips", 

• • • 

"destroyed by a deathwatch beetle (Xestobium rufovillosum)" , 
"dead at a medieval pulpit that's carved from a single oak", 

• • • 

"quite armless erected as the Venus de Milo", 

"dead in the miniature village", 

"gently extracted by a cold press juicer", 

• • • 

"murdered at the suggestion of a singing elevator voice", 
"imprisoned in a vast wicker figure with goats and burned alive" 

































https://github.com/matthewberryman/midsomerplots 


var village_group = [ ... ]; 

var angry_at = [ ... ]; 

var threatened = [ ... ]; 














https://github.com/matthewberryman/midsomerplots 


var rands = 
Mai .floor 
Mai .floor 
Mai .floor 
Mai .floor 
Mai .floor 

] ; 


( mt.random() 
( mt.random() 
( mt.random() 
( mt.random() 
( mt.random() 


* murdered p erson.length ), 

* cause_of_death.length ), 

* village_group.length ), 

* angry_at.length ), 

* threatened.length ) 


return murdered_person[rands[ ]] + " is found " + 

cause_of_death[rands[ ]] +" . Suspicion falls on the village " + 
village_group[rands[ ]] + ”, angry that " + angry_at[rands[ ]] + 

" might threaten " + threatened[rands[4]] + 








Cheap Bots Done Quick 

https:/ / cheapbotsdonequick.com 


f DUNSONs & DRAGGANs 

! ©DUNSONnDRAGGAN 

You are in the Office department. 

You hear a noise and spin round; it's a 
HENRIKA scavenging the remains of a 
disassembled NUMERAR 

10:28 AM -29 Nov 2017 



Thinkpiece Bot 

©thinkpiecebot 



We Don't Have a Bot Truthers Problem, We 
Have a Jerry Seinfeld Problem 


5:11 PM - 1 Dec 2017 



unicode_garden 

©Unicode garden 



1:33 AM - 30 Nov 2017 


































http://tracery.io 



" S [ 

"#color.capitalize# #animal.s# are #often# #mood#.", 

"#animal.a.capitalize# is #often# #mood#, unless it is #color.a# one." 

], 

"often": ["rarely" , "never" , "often","almost always" ," always" , "sometimes "], 

"color": ["orange" f "blue" f "white" f "black","grey","purple","indigo", 

"turquoise"], 

"animal": ["unicorn","raven","sparrow" , "scorpion","coyote" , "eagle" , "owl ", 

"lizard","zebra","duck","kitten"], 

"mood": ["vexed" , "indignant" , "impassioned" , "wistful" , "astute ", 

"courteous"], 

"natureNoun": ["ocean","mountain","forest","cloud","river","tree","sky", 

"earth","void","desert"] 
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Horse ebooks 

@Horse_ebooks 


Everything happens so much 

1:23 PM - 28 Jun 2012 











































@thegrugq / @thegrugq_ebooks 


the grugq 


the grugq 



©thegrugq ebooks 

Give a man an Oday and he'll have access for 


Check the credits in Android 0 are actually 

a day, teach a man to phish and he'll have 


relevant here, but only to the dozen or so 

access for life. 


people can actually secure their accounts. 

11:35 PM - 6 Feb 2015 


8:17 AM-28 Nov 2017 



the grugq 

@thegrugq_ebooks 


Possibly the best off the shelf solutions to 
monitor and cyber secure that basket!!! 

10:02 AM -27 Nov 2017 







































@erowidrecruiter 


'TtV Erowid Recruiter 


Erowid Recruiter 

©erowidrecruiter 


.. ©erowidrecruiter 

1 feel the essence of pure energy, and life, 


I’m still somewhat concerned that, yes 

and oxygen, flowing through each and every 


Virginia, it’s been 26 goddamn hours and I’m 

one of the Google Maps Javascript APIs 


a Technical Recruiter. 

3:20 PM - 6 Feb 2017 


10:19 AM - 13 Feb 2017 



Erowid Recruiter 

©erowidrecruiter 


Clojure Engineers with the underlying 
consciousness grid, god, gaian supermind, 
universal consciousness, whatever you want 
to bring 


6:03 PM -28 Jul 2017 







































wintnation 



Wint Nation 

©wintnation 


Pauline Hanson’s One Nation party is not just 
a smoke and mirrors job to trick the 
Australian people. 


9:09 PM - 25 Sep 2017 



Wint Nation 

©wintnation 


terriffic news lads, i invented my own ebola 
virus by lying face down on my plans to help 
reform the family law court. 


4:43 AM - 22 Jul 2017 



Wint Nation 

©wintnation 


Silly Dave, there is no consensus in science 


9:15 PM - 12 Aug 2017 



Wint Nation 

©wintnation 


Until we get tough on this nothing will 
change and we will see a sharp decrease in 
recreational nudity 


5:19 PM-17 Jul 2017 
















































Pauline Hanson mo 

@PaulineHansonOz 


END MASS MIGRATION | Report shows 
majority of Australians support 
@PaulineHansonOz call for lower migration. 

#auspol 



Report: Voters Say Country Is Full, Support Partial Muslim Immigration Ban 

In a blow for multiculturalism. a majority of Australian voters believe the country is 
full and want Muslim immigration halted. 

breitbart.com 




5:17 PM -26 Oct 2017 































wint 

@dril 

"im not owned! im not owned!!", i continue 
to insist as i slowly shrink and transform into 
a corn cob 


4:20 PM - 10 Nov 2011 




































who the is scraeming "LOG OFF" at my 
house, show yourself, coward, i will never log 
off 

11:36 PM-15 Sep 2012 



































Habeas Corp 
































https:/ / apps.twitter.com 


Application Management 


* 



Twitter Apps 


Create New App 




































https:/ / apps.twitter.com 



Create an application 


Application Details 

Name * 


Your application name. This is used to attribute the source of a tweet and in user-facing authorization screens. 32 characters max. 

Description * 

Your application description, which will be shown in user-facing authorization screens. Between 10 and 200 characters max. 


Website * 










































https:/ / apps.twitter.com 
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My Demo App 

Details Settings Keys and Access Tokens Permissions 


Test OAuth 


Application Settings 

Keep the "Consumer Secret" a secret. This key should never be human-readable in your application. 

Consumer Key (API Key) i WB&c-Z-Tizrz- 


Consumer Secret (API Secret) 











































https:/ / apps.twitter.com 


Your Access Token 

You haven't authorized this application for your own account yet. 

By creating your access token here, you will have everything you need to make API calls right away. The access token generated will be assigned your 
application 's current permission level. 


Token Actions 

Create my access token 




































https:/ / apps.twitter.com 


Your Access Token 

This access token can be used to make API requests on your own account's behalf. Do not share your access token secret with anyone. 

Accgss Token 

Access Level Read and write 

Owner jeamland 


Owner ID 








































import twitter 

api = twitter.Api(consumer_key=CONSUMER_KEY, 

consumer_secret=CONSUMER_SECRET, 

ac c e s s_token_key=ACCES S_TOKEN, 

access_token_secret=ACCESS_TOKEN_SECRET, 
tweet_mode=' extended' ) 

max_id = None 
for x in range (11 ): 

statuses = api.GetUserTimeline(screen_name=user , count= , 

max_id=max_id , include_rts=False ) 

if not statuses: 
break 

max_id = statuses[ -1 ].id - 1 

# Do something with statuses 

















https:/ / developers.facebook.com/tools/explorer/ 


facebook for developers 


Products Docs Tools & Support News Videos 


Search 


My Apps ▼ 


Graph API Explorer 


Application: I?] Graph API Explorer ▼ 


Access Token: 


GET* -» / v2.11 - / 


Get Token ▼ 


► Submit 


Learn more about the Graph API syntax 





















































import requests 


url = ' https://graph.facebook.com/v2.ll/%s/posts' % PAGE_NAME 
while url: 

r = requests.get(url, params={ 

'access_token' : FACEBOOK_ACCESS_TOKEN, 

'fields 1 : ' id,created_time,message , 

}) 

data = r.json() 

# Do something with data 


url = data[' paging '].get(' next' , Tone) 















Remove extraneous content 



































Keep timestamps for topicality 



































Play with your content mix 































































































State — some number of words or tokens 


Look-up table for each state containing weighted possibilities for the 
next word 

The next selected word becomes part of the state, the oldest word or 
token in the state is removed 

Repeat until desired end state is reached 
















































































am 


BEGIN BEGIN I 


seem 










































ecstatic 


BEGIN BEGI I am } ro o t 









































BEGIN BEGIN I 


END 


am Groot 








































I am Groot 

































Wint Nation 

@wintnation 


I met with the best of all the political parties, 
but if you do one more weird trick youll die 

10:38 PM -29 Apr 2017 




































Sen. Malcolm Roberts 9 

©SenatorM Roberts 


Replying to ©jjmcghie 

I met with the best of the CSIRO and they 
couldn't provide empirical evidence of 
human causation 

4:00 AM-26 Jan 2017 



Colin Tincknell 

®tinka133 


Our PHON corflutes are the best of all the 
political parties, but if you give them back, I 
get Pauline to sign them! 


2:31 AM - 1 Mar 2017 



doctor: i can say with absolute certainty that 
if you do one more weird trick youll die 
me: CAPTAIN TIGER'S Miracle Corn. LOok it 
up bastard 

6:43 AM - 16 Feb 2015 


1 



Wint Nation 

@wintnation 


I met with the best of all the political parties, 


but if you do one more weird trick youll die 


10:38 PM -29 Apr 2017 

































































0 0 



































































Rings & Responsibility 



































markovify 

https://github.com/jsvine/markovify 












































import markovify 


CORPUSES = [ 

'austen-corpus.j son', 

'tolkien-corpus.j son', 

] 


models = [] 


for filename in CORPUSES: 

with open (filename) as data 
models.append(markovify 


Text.from j son(data.read())) 


model = markovify.combine(models) 



i in range ( ): 
print (model.make 


sentence()) 









Its long, damp passages, its narrow way between pine-clad walls; and as 
they came back painfully to wakefulness. 

I would choose to answer." 

He evidently tried to get it out of my having never yet imagined opened 
out before him, Mr and Mrs Clay found him all alone for the bit of a 
French privateer was at the broken peaks and high enough above the wind 
had been found in the shadow. 

They proceeded in her conduct, and removes all the landings held now by 
the approach to coarseness, however, or any particular compliment to you 
very soon. 

"Sir Thomas," said Edmund, "and one of my own voice speaking from the 
world. 





































import markovify 


SOURCES = [ 

'austen-corpus. 

'tolkien-corpus. 

] 


i 


for source in SOURCES: 

model name = source.replace ('. txt' , ' .json E ) 


with open (source) as data: 

model = markovify.Text(data.read (), state size= ) 


with open (model_name, ' w' ) as model_file 

model file.write(model.to json()) 










"Undoubtedly," replied Darcy, to whom this was chiefly addressed, 
replied, "I believe I have, but this is the One, and they became 
Ringwraiths, shadows under his great Shadow, his most terrible servants. 

The commendation bestowed on him by Mrs. Dashwood, and he found in it a 
source of immovable disgust. 

She had been satisfied to hear and satisfy his inquiries after their 
mother were respectful and attentive. 

Crawford called the next day, in spite of her being amongst them by her 
niece; and being so soon to leave Mansfield, and on this being evident, 
she was so fond of it, it seemed to him that anybody could live in 
Devonshire, without living near Dawlish. 


"And I am changed also; for I am sure he is in love. 







































Part-of-Speech tagging 



































import markovify 
import nltk 

class POSifiedText (markovify.Text): 
def word_split ( self , sentence): 

words = re.split (se_ .word_split_pattern, sentence) 
words = .join(tag) for tag in nltk.pos_tag(words)] 

return words 

def word_join( sel , words): 

sentence = .join(word.split ('::') [ ] for word in words) 

return sentence 












import markovify 

from posi iedtext import POSifiedText 

SOURCES = [ 

'austen-corpus.txt' , 

' tolkien-corpus.txt , 

] 

for source in SOURCES: 

model_name = source.replace(' .txt ' , ' .json' ) 

with open (source) as data: 

model = POSifiedText(data.read (), state_size= ) 

with open (model_name , ' w' ) as model_file: 

model file.write(model.to json()) 










import posifiedtext 

CORPUSES = [ 

'austen-corpus.j son ' , 

'tolkien-corpus.j son ' , 

] 

models = [ ] 

for filename in CORPUSES: 

with open (filename) as data: 

models.append(posifiedtext.POSifiedText.from_json(data.read())) 

model = markovify.combine(models) 

for i in range(5): 

print (model.make sentence()) 









"I am so glad.—Quite out of Mordor, and more diffused intimacies little 
groups were formed, and her mother and sisters, never seems to me in 
these darksome holes surely? 

Miss Bennet will receive every possible vice, going on a recovery in her 
at that moment with his strangely sharpened hearing Sam could go there 
himself. 




































Mr. Darcy smiled; but no attitude could give her ease, she hurried away, 
leaving the hobbits where they lay. 

There are many evil and unfriendly things in the country you see the 
house in the neighbourhood, and the regard which his social powers had 
gained him in the twilight. 

There's been a strange customer asking for Mr. Baggins would be let 
through. 

If now and again it lifted its head slowly, turning it right back on its 
long neck was ever turning this way and that; and Frodo knew with 
certainty and horror that among the many crossing paths, although left 
to themselves they would soon grow more alike. 

He had no reason to suppose he will make an offer to Lizzy, and she will 
force you to do justice to himself; but when his eyes beheld the 
Mountain and set out, thinking no more of her than her house would hold. 





































Quality Control 

































Benno and the Terrible, Bad, No Go... 

@jeamland 


What is Machine Learning? Easy! Machine 
Learning is how you automate your biases so 
you can think about them even less. 

10:12 AM-7 Oct 2017 




































Generation 






















































Backfed Input Cell 
Input Cell 
Noisy Input Cell 
Hidden Cell 

Probablistic Hidden Cell 
Spiking Hidden Cell 
Output Cell 

Match Input Output Cell 
Recurrent Cell 
Memory Cell 
Different Memory Cell 


A mostly complete chart of 

Neural Networks 

€2016 Fjodor van Veen - asimovinstitute.org 


Perceptron (P) Feed Forward (FF) Radial Basis Network (RBF) 



Deep Feed Forward (DFF) 


Recurrent Neural Network (RNN) Long / Short Term Memory (LSTM) Gated Recurrent Unit (GRU) 




Auto Encoder (AE) Variational AE (VAE) 



Denoising AE (DAE) 


Sparse AE(SAE) 













































oogle 


python train Istm language modelling 



All Videos News Images Shopping More 


Settings Tools 



About 146,000 results (0.55 seconds) 

examples/wordJanguage_model at master • pytorch/examples • GitHub 

https://github.com/pytorch/examples/tree/master/word_language_model ▼ 

Word-level language modeling RNN. This example trains a multi-layer RNN (Elman, GRU, or LSTM) 
on a language modeling task. By default, the training script uses the Wikitext-2 dataset, provided. The 
trained model can then be used by the generate script to generate new text. 





















o 


This repository Search 
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£ +- n 


0 pytorch / examples 


Branch: master ▼ 


examples / wordJanguage_model / 



adamlerer committed with soumith Swap PTB for Wikitext-2 (which is open access) 


data/wikitext-2 


U README.md 


D data.py 
D generate.py 


=] main.py 


D model.py 
D requirements.txt 


Swap PTB for Wikitext-2 (which is open access) 
Swap PTB for Wikitext-2 (which is open access) 
Refactor language modeling example 
Swap PTB for Wikitext-2 (which is open access) 
Swap PTB for Wikitext-2 (which is open access) 
Handle tied + dimensions mismatch (#124) 


doc 


<•> Watch ▼ 


162 


★ Star 
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y Fork 
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python 

epoch 

main 

1 

.py —data corpus —epochs 
200/ 1219 batches | lr 

6 —tied 

20.00 ms/batch 

2020.79 

loss 

8.18 

l 

ppl 

3559.39 

epoch 

1 

400/ 

1219 

batches 

lr 

20.00 

ms/batch 

1987.56 

loss 

7.03 

l 

ppl 

1135.43 

epoch 

1 

600/ 

1219 

batches 

lr 

20.00 

ms/batch 

1969.26 

loss 

6.57 

l 

ppl 

716.19 

epoch 

1 

800/ 

1219 

batches 

lr 

20.00 

ms/batch 

1995.88 

loss 

6.33 

l 

ppl 

559.07 

epoch 

1 

1000/ 

1219 

batches 

lr 

20.00 

ms/batch 

1985.88 

loss 

6.18 

l 

ppl 

484.82 

epoch 

1 

1200/ 

1219 

batches 

lr 

20.00 

ms/batch 

1964.92 

loss 

6.07 

l 

ppl 

434.61 


end of epoch 


time: 2652.19s valid loss 5.94 valid ppl 379.68 



end of epoch 6 


time: 2630.23s 


valid loss 5.41 


valid ppl 


224.03 


$ 








































































$ python generate.py —data corpus 
| Generated 0/1000 words 
Generated 100/1000 words 
Generated 200/1000 words 
Generated 300/1000 words 
Generated 400/1000 words 
Generated 500/1000 words 
Generated 600/1000 words 
Generated 700/1000 words 
Generated 800/1000 words 
j Generated 900/1000 words 

$ 


-checkpoint model-base.pt 













































And the one evening the letter now itself is beyond first many months 
since an gollum has wished, that there is some charm and spirit which 
will be absolutely searches in 
your stable." 

said Strider too prosings 

"And I pretend, for there ended the same part, of sleep it, of regard at 
the uneasiness of Harriet's feelings. 

But with his hope the flood had served the Man that Hasufel was come! 

I will take my Gollum, what is at 'Give me but an old quarters. 

Anne could probably have waited on what had passed, when the first stage 
beyond the discovery, was praise, their satisfaction undetermined, 

Elinor was the Bombadil: made judiciously over the day of her family, 
remarkably sluggish and proposition, on the subject with 





































Timings 


State Size 2, no PoS 

9s 

State Size 3, no PoS 

10s 

State Size 2, PoS (nltk) 

68s 

State Size 3, PoS (nltk) 

66s 

State Size 2, PoS (spacy) 

610s 

State Size 3, PoS (spacy) 

850s 

PyTorch, laptop 

16300s (~4.5hrs) 

PyTorch, p3.2xlarge 

790s 





























Part-of-speech tagging, redux 



































He was really obliged to dine at the party on Frodo's face on a strong 
hill to admiring himself, and he soon turned aside. 

Frodo was soon astonished to understand that Mr. Knightley had 
discovered his good, and his eye had been monasteries. 


How Lady Legolas wondered that this land was wounded. 
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Merry Christmas and have a political leftist 
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